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- A SURVEY OF TEST VALIDATION STUD* CQSTS 



ABSTRACT ' • 

This survey provides informatics on total costs, coet^ elements, and ; other 
related background aspects of validation research* Cost element-? covered 
were staff time {btolcen into researcher ancT clerical, subject matter expert, 
and research participant), overhead, travel, and Miscellaneous costs* Infor- 
mation is reported for 21 studies or groups of studied* *In round figures, 
costs ranged from $24,000 to $673,000 for criterion-related studies and from 
$8,000 to $1,245,000 for content Validity studies/- The sources of information 
were in most cages the recollect ions or records* of persons having knowledge of 
each*study. The accuracy of the information is unknown* 




A' SURVEY OF TEST VALIDATJON STUDY- COSTS* 



j * - This survey of the costs of 
conduoting test validat ion research 
showed a wide range of expenditures^ 
The rounded costs for 1 individual 
criterion-related studies ranged 
from $24,000 to $673*000. The costs 
of content validity studies ranged ' 
from 58/000 to $1,245,000. The work 
done 4 varied widely in scope, com- 
plexity, and cost elements covered. 
The intent of the survey was to pro- 
vide information on several cost 
elements and cost-related aspects 
of research. Records of the types 
of detailed 'cost figures surveyed 
are" not routinely kept by, research 
personnel. The sources of informa- 
tion for the survey were, in most 
cases,: £he recollections and records 
o£ persons having knowledge of each 
study. Several respondents stated 
that their recollections were only 
"ballpark 1 * estimates subject to 
considerable error. In- two cases 
financial information was avail- 
'able, and in, another a written, 
cost report was prepared by a prin- 
cipal researcher. 



The Scope of the Survey 

The basis ot the surveywas' 
a 1975 report of total costs for 30 
validation studies done in the 
public sector that was compiled 
for the Equal Employment Opportu- 
nity Coordinating Council (United ' 
States Civil Service Commi ssibn r 
Note 1). The principal researchers 
or other knowledgeable persons 
of 17 of those studies furnished 
cost data. In addition, data was 
obtained frpm eight studies that 



were known to^the author or discov- 
ered through referrals from persons 
having knowledge of ongoing validity 
research, or found by contacting 
organizations involved in test val- 
idation. FOr some studies costs 
were available in terms of the cost 
6f <a larger group. These studies 
were treaped as a group in the' sur- 
vey, t 

In general, the studies*cov- 
ered represent work in several 
research situations, or* combina- 
tion s\of situations, carried out by 
,or for state jurisdictions or 
military or civilian sectors of 
the Federal governments Seven 
studies were conducted by consult- 
ants drawn f ronrunivers^ities or 
private firms. At leasts six pro- 
jects were funded by Federal grants. 
At least two studies were intended 
as pilot projects or training 
expediences. Four were components 
of a larger construct validity 
design. At least one was conducted 
in an atmosphere of litigation. 
Six were examples of cooperative 
research involving more than one 
jurisdiction or organization. 

With respect to research 
strategies, both content and cri- 
terion-related studies were; covered. 
Criterion- related studies included 
both predictive and concurrent 
designs. The cost of construct 
validation was not specifically 
addressed by this survey. Four of 
the criterion-related studies cover- 
ed were, however, conducted in supP 
port of the construct validity of 
one written test. The total cost 



The information contained In thia survey was gathered by telephone or personal contact 
with principal researchers or other persons^ he /I tig a ftetalied knowledge of recent valida~ 
tlon studlea. Because itany persons were Reluctant to mention Individually Identifiable 
cost figures, all studies have beta deacribed In general terms. Although the names of 
persons auryeyed have been-lcept anonymous, the cooperation and gracious aaslttance of these 
persona la gratefully acknowledged. - 
: * ft 
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of these studies (see tables i and* 

• 2, entries 11, 1?, 13 and 14) was* 
5687,000 rounded 1 to the nearest 

1 thousand. The scope "of this survey 

• 'f3oe£ not cover a representative sam- 
ple nor d6es it contain- a compila- 
tion of 1 typical or average study 
;costs. Studies were selected on the 

basis of the availability of cost 
figures. 

t i 

The survey addressed four 
elements of research activity; 
the validity paradigm, 30b analy- 
sis, predictor development, and 
criterion development, .Six cost 
elements wer^e coveted : researcher 
and clerical tirfe> subject-matter- 
expert time, research, participant 
time, overhead, travel, and miscel- 
laneous costs such as printing, 
da^a - processing, and auxilliary 
consultant costs; In some cases 
researchers were unwilling or / 
unable to calculate more than 
overall .figures for their -costs. 
No judgments were made concerning 
the lechAical adequacy or potential 
utility *of any study. 

Source of -Information 

J / ' 

,For each study, a principal 
researcher or project manager was 
interviewed, either by telephone 
or by personal contact. In two * 
cases, referral was made to-a 
grants manager who had knowledge 
of , both financial and. background 
aspects of the project. 

Eac-ft contact persoa was inter- 
viewed to obtain background infor- 
mation concerning (a) "the starting 
and ending dates oT v ,the study, (b) ! 
the validity patadi^m "used, (c) 
.whether the work included 30b analy- 
sis, Jd) if a *predictor had been de- 
veloped, (e) whether; criteria had 
b^^n? developed, and (f) t^he final 
dumber of" research participants in 
the study* . Persons were also asked 
if* they had access* to or could 
construct cb^tf f i<jut'es for the 
cost elements, add^^sed by the 
survey. \EinAlly, each* person was 
asktfd if "he or sh^ ka& knowledge 
of f othet studies, for Hhich cost 
figures might be"" available. It 
wa£* agreed that ill' figures were 



to be kept anonymou^. ^ Because of 
the agreed-upon anon^feityj data 
sources were not' referenced. Never- 
theless, /soj^e persons contacted^* 
were reluctant to reveal cost f ig- t 

* ures. In addition, one sstudy for 
which cost figures were available 
was omittfed at the request of the 
principal researcher, who consid- 
ered the costs to be atypical*, be- 

- cause of the complexity and pio- 
neering nature of the Research. 
The tota? cost of 'this research was 
in the mid-orange *of the studies 
analyzed here. For ten' studies 
a technical report or other writ- 
ten documentation was available. 



. Tlxe Costs Determined and Their 
Accuracy 

For all studies, the largest 
single cost element was researcher - 
and clerical time. Inconsistent 
interpretation of this- cost ele-* 
ment affected its magnitude in . 
individual studies, in some cases, 
it was interpreted as the total " 
amount of researcher and clerical, 
time budgeted to the stydy. 
In other cases it was interpreted . 
as the amount of 'time specifically 
?pent on research duties only and 
did not include time spent on any 
related duties. In many instances, 
interpretation of this cost element 
was unclear. 

Subject-matter-expert time ^ 
was defined as "the cost of persons 
from the studied occupation partic- 
ipating, in the research project 
other than as research subjects. . 
Interpretation of thisxogt element 
was inconsistent, *and judgment was 
necessary to separate subject- 
matter-expert time frofo researches, 
and clerical time. In two studies, / 
researcher and clerical time includ- 
ed both consultant costs and costs 
of personnel from the organization 
engaging the consultant. In other 
cases, organizational personnel 
costs were included as subject-mat- 
ter-expert costs by the source per- 
son. Here the decision was that 
such persons were functioning pri- 
marily as advisors on subject matter 
cather than as managers or co- 
operative researchers in the study. 



Research participant time was 
the second largest cost element, > 
although this element and subject- 
matter-expert time are ones^en* 
^eraJ^y omitted from total figures 
because,, they, are not costs directly 
incurred by researchers. They are, 
however, costs borne by the organ^x 
zation conducting or, contracting ^ 
for the research. # Participating 
employees must be* released from 
their regularly assigned duties. 
When participating employees are 
reassigned, their regular dutres 
'must be carried out by others or 
remain undone. .Resulting morale 
problems, failures to meet 'dead- 
lines, work slowdpwns,*and other 
such problems may ail contribute to 
the actual costs of research appro- 
priation ^ of employees. Therefore 
estimates of the cqst of 1 research- 
participant and subjedt-matter- 
expert time based on salary or 
salary plus overhead may be low* 

, Overhead was another relative- - 
ly large cost element. In some of 
the studies reported, the inclusion 
of overhead or its amount was * 
unclear. Overhead was defined to 
cover a variety of cost factors, 
ranging from fringe benefits only * * 
t6 estimates' covering building 
space, heating, and other such 
physical necessities. Overhead 
was reported to be as much as 100 
percent of staff time c<5st$. 

Table 1 shows the background, 
information collected on-each 
study. Table 2 shows costs of 'each 
study broken down into cost ele- 
ments.' Notes for each table are 
appended. 

** 

/ 

Discussion and -Conclusion 

* Thfe complexity of the re- 1 
search strategy and the, number and 
kinds of predictors used definitely 
affected the amount of sta£f time 
required to carry* out e£$h piece 
of research. In comparing costs 
among studies it should be noted 
that several other ' factors'fliay in- 
fluence research costs, ^he exist- 
ence of job analysis, its bomplexity, 
and the development of multiple cri- 
terion instruments also *f feet ' costs, 



as does the amoJnt of work required 
to develop and refine a predictor 
instrument. Clearly the number 
and salary level of research par- 
ticipants and subject matter ex- 
perts are significant cost tactots. 
Other aspects of" the research 
process may also exert aninflu— 
ence. The anticipation or the 
presence of litigation and the ■ 
presence d£ the 1 Uniform Guidelines 
on Employee Selection (USEEOC, 
USCSC, "uSDOi, 'fi "USDOJ, i978) would 
result in, more attention to/exten- , 
sive documentation ^nd to efforts 
to control for such variables 3s 
race, sex, ^nd ethnic group, and 
would therefore increase costs. 
Studies carried out by several co- 
operating organizations and ".those 
using a research pro}ectas a 
training device would also involve 
more expenditures.* 

Because itemized cost f itjures 
are ncft routinely kept by research- 
ers, data were, for the most part, 
drawn from ^collections and records 
of research personnel and wer*e io 
some cases no more.tfiah rough esti- 
mates. > When a- particular piece of 
research had been supported t>y a 
Federal grant, \budget documents were 
often available, but may have at 
times underestimated 'actual research 
6osts. The obtained figures do, 
however, represent some possible,, 
cost figures and do clearly demon-- 
strate the wide range of costs pbs— - 
sible in conducting validation -re- * 
search. Such figures m could be help- 
ful to someone planning a research 
project. " * > ' t 1 

Ideally, a standard reporting 
format would include a number of - 
defined <;ost elements. ,1 would 
recommend the following list: 

I. Staff Time Cost&. % 

* > 

A* Research t^aro Jjeadej: 

B. Research t£am members:** 
Ml nonclerical project p^rsbnnel^ 
other than the team leader who are 
primarily concerned With sciehtifip* ] 
aspects of the-worft ^ 

C. cierioal personnel- ^ ' : * 
D* Administrative personnel* 1 " + h * 

when research is done- primarily /by* ^ 
a consultant this t element, is t^ - 



TABLE 1 

7 

.Background Information for the Studies Surveyed * 



Study 



Type 



Development Development 
Job of * of 

Date Analysis Predictor Criterion N 



I. Police Exam A 



Criterion-related 1U76- 
, (predictive)* . 1978 



No 



Yea 



Yes 300 



2. Poll.ce Exam B 

3. • Police Exam C 
* . Police Exam D 

5. Police Exam E 

> \ 

6- : * Firefighter 
Exam A * 



7. 



FijefjJ^ter 
Exaifr B 




.'Firefighter 
\Exam C ^ 

Transit* Operator 



Content 6 
Criterion-related 

Criterion-related 
(concurrent) 

Criterion-related 



Criterion-related 



Content & 
Criterion-re lated 

Criterion-related 
.^concurrent) 

Criterion-related 



1976: 
1979 

19721 



1974- 
1978 

1972- 

1973 ' 

1975- 
1979 



.1976 



1968- 
1979 



Yes 



Yes 



Yes 



Ytfs 



Yes 



1972 Yes 



Yes 



Yes 



/ 



Yes 



No 



Yes 



No 



Yes 



No 



Yes 



Yes 



Ye,s 300 
Yes ] 275 

Yes 1,700 

Yes 802 

.Yea 214 

Yes 328 

Yes 264 

Yes 1,471+ 



TO.* 



11. 
12. 

13; 



% Group of 3 Criterion-Delated 

s tudiea; * \ f v - y 1 * , v 

We^tlcal Techni- \" / ^ > 

clan^Xnvent^ry^ * * J' 

Manager, Carfco- + * v 
^bephic/Technt- - 
clan ; , 

Customs Ins^ctor Criterion-related 

Social Insurance ^ Criteftion-pjlated 

Claims Authorized <- f " 

stud$r ^ : ;\ . 

Social Insurance /^fcri tgvion-jrelated 
Claims Atft'horl2er 
Study B a * 



1968- 
1973 



Yes 



No 



Yes 1,400 



1975- 
1977 

1974- 
1975 



1975- 
1977 



Yes 



Yes 



Mo 



No 



No 



Yes 



No 1 



190 

'Z53 

175 
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TABLE 4 (Continued)'*; t 



Study 



Type 



Date 



Development Development 
Job of of * 

Analysis Predictor Criterion N 

^J2 . 



14. 



15. 
1 ' 



16. 



. 17, 



Internal Revenue 
Officer - — 
Study g 

Average study 
cost (based On ,5 
studies done In 
one juris- 
diction)* 

Group of 6 
studies conduct- 
ed in one . 
jurisdiction 

40 military 
occupations 



18. Military 
technical 
training 
■ stjcce'ss 



Criterion-related 



Criterion-related 



Criterion-related 



Criterion-related 
^ (predictive) 

Cri terlon~related 



1975- 
1977 



1975- 
1478 



1973 



1976- 
1978 

1977 



Yes 



Yes 



No 



No 



Wo 



No . 



Yes 



Yes 



No 



No 



Yes 



Yes 



Yes 



No 



No 



75- 
300 



40- 

107 



5,000 
A3, 985 



19. 46 Public 
Service 
Occupations 

20. State Police 



21. Clerical, exam 
Multl-juris- 
^ fictional study. 



'Content 



Content 



Content 

/ 



1972- 'Yes 
1974 



1975* Yes 



1973- 
1976 



Yes 



Yes 



Yes 



Yes 



Hot Uoavall- 
appllc. able 



Not Unavail- 
appllc. able' 1 

t Not *300- 
appllc*' - 400 



fiote* Appendix 1 contains notes to entries fn Tables 1. 



■ a t 

This study had three parts; a concurrent sfudy having -an s of* 1 113 males, a. • 
predictive study having an N of 358 males, sand ^differential study of females for which the 
// was unavailable. > - 



ERIC 



TABLE 2 



Dollar Cost of Studies Surveyed, by Cost Element 



Other 



Study 



( Staff Time ■ » 

Researcher^ Subject Research 
and matter , "parti- 
clerical expert clpant Overhead Travel 



Miscel- 
laneous Total 



1» Police Exam A 

2. Police Exam B 

3. Police Exam C ' 

4. Police Exam D 

5* Police JSxam E 

^..Firefighter 
, t Exam A 

7. Firefighter * 
fixam B - 

8. Firefighter - ; 
; Exam C 

9. Transit 
Operator \ 

10. Grout* of 3 
Studies: 
Medical Techni- 
cian; Inventory 

- Manager; Carto^ 
graphic Technic 
cl « . " \ 

11. Customs 
Inspector 

12.. Social In- 
surance Claims 
Aut Ho riser % 
"Study A 

13. gocial In- 
* sura nee Claims 
Authorlser 
Study B 

J 4* Internal - 
Revenue'. 
* Officer 



51,500 3,000 , 30,380 

1^0,000 34,880 ^ 27,150 

30,000- 10,000 16^500 

132,877 1 , . m 127^500 



3,0,000- 10,000 v 19,700 



123,000 15,000 i0,085 



295,514 
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8 VI 59 



592,300" 9,536 200,256" 
■ * • * 

167,813 57,184 31/803. 

150,772 % 5,089 29,008 

35,02* " 2,067 - 10,430 

80,898 ".. 19,228 32,542' 



12,852 
18,609 

5,000 

85,964 58,620 



265,345 14,057 33,884 5,355 none 



5,000 

199,003 62,295 



61,200 
10,181 



100 



no 



800 98,532 
230, 639* 
61,500 
55,698 460,65$ 
70,000 

5,962 a 334,603 

64,700 

150,085 

26,050 667,021 



802,092 
11,900 278,881 



5,014 13,403 203,288 



^ 5,179" 52,084* 



7,432 * 11,900 152,000 



i 



TABLE 2 {Continued* 



Staff Time 



Other 



Study 



Researcher Subject Research 
and matter > parti- 

clerlcal expert clpant 



Miacel-, 

Ove r h e ad Tr a ve I I ane oua* 



Total 



15. Avg. study 
cost(based 
- on 3 studies 
^done' In one 
Jurisdiction) 

J6. Group of 6 
t Studies 

conducted 

In one 

jurisdiction* 

17. 40 military 
occupations 

18. hiUitary 
technical 
training 
success 

19. £6 public 
* service 

* occupations * 

20. State Police 

21. Clerical Exam 
mult'l- 

jurisdictional 1 
study 



,33,500 



None 3,100 



, 200 3^860 ^ 40,660 



120,000 

50,000 ■ None 
* 

666,302 

152,846 152, 846. 



13,000 



80 12,D00 145,080* 



None 



6 t ,525 



V 

56,989 



'50,500 



672,827' 



1,358 364,039 



1,245, 00p 



105,000 



Note* Appendix 2 contains notes to entries In Table 2. Blank spaces indicate that , ; 
Information on the cost element was unavailable, 
a 

The average cost of one study was approximately $267,833* 
b < * - * „ 

The average cost of one study wss approximately $24,-180. K ^ 

The average coat per occupstlon was $7,913, 
d " 
Only a total cost figure wss available; Information; on separate cost elements could 
not be obtained. * ^ v - * • * •* 
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cost of personnel^contracting £or 
and managing the work;- In *co- 
operative studies it .would be the 
cost of personnel who primarily 
attend to administrative rather 
thag scienti fic %£pects of't^e work 

. E. Research participants; 
persons from t#ie occupation under 
study to whom research instruments 
had been administered, or supervi- 
sors who supply performance appr.azs- 
-als ^L., -.- ' ^ _ 

-F. * Subject matter'%xperts; 
persons from the occupation under - 
study who supply advice on subject 
matter or give administrative sup-; 
port v 

*^ f 
II.- Travel 

* h 

nf. Overhead:* space t and util- h 
ities costs, t fringe benefits, Office 
equipment and supplies 



IV. ADP costs: computer process- 
ing and the time of " programmers and 
other ADP personnel- 

V. Miscellaneous consultant- 
costs: costs for ^ oonsultartt v^en 
jonly a small segment of the scien-* 
tific or technical aspects 6f the 
work i^done under contract 

1 VI. Miscellaneous costs such 
»a k s printing, mailing, or forms de- 1 
sign x - 



Systematic collection of 
such data would 'be useful to re* 
searchers in many instances. Such 
fUgurgs "Would 'provide the researcher 
with a solid ^basis' for making budg- 
etary decisions. Undec the. Uniform 
Guidelines* on Employee Selection^ 
(USEEOC ht al., 1978) usera of se- 
lection procedures will be required 
to prepare^ extensive and often cost- 
ly documentation. Systematic com* 
pilatidn of research costs would 
indicate areas where^costs- could 1 * 
feasibly be reduced. 

A researcher might keep such 
figures bn a monthly basis, and, 
at the end of a year 4 , *be betted , 
able to analyze his or Mr work. 



For instance* the cost of research 
participant time could be- investi-, 
gated "in, terms of statistical t 
power needed and the .cost pei; 
participant. Ii>formatioo gained 
.could also be usetj to assess the* 
utility of the research product 
and to compare the jcost -effective- 
ness of different approaches to ,se- 
lection. For example,, selection by 

computerized tailored testing 

coul& be compared "To selec t i on * 
by a standard, content valid papeif^ 
and^pencil test. . 

Togo a step further, if costs 
w^re compiled 1 in a standardized * , 
tashion, figures from a number of- 
studies could' be pooled to yielc* av- 
erage cost figures fqr similar - 
types of research. If such stan- 
dardized data were centrally grouped 
-according to similarity of research 
design, representative cost element 
figures for groups of similar stud- * 
ies could be prepared. These £ig-> 
ures, could be, stored and updated by 
means o£ ADP and would provide ^n- * 
formation useful to persons ^min- 
istering »and planningnew research 
or to persons managing or applying 
for research grants. 
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ABJJENDIX ^ 



Notes to Entries In Table 1 - J 



1. Multiple predictors, were used, some developed as part of the research. Criteria c 
were tral'nlng*Su&cess, work simulation and field ratings. Information sdurc^:, 
records and recollection of project director. > j 

2. Research conducted by a consultant. Information source; recollections of [ 
consu£tan£ and police department project manager. " i 

3. Research conducted by a consultant. Study w^as tifrne for.snd managed by a nultl- 

\j jurisdictional consortium. Information -sources recbllectlon and redords of project 
manager *and' project "personnel. 

■ [ . J . \ : ■ ' ■ * . - 

4. Study was conducted by a university research group and involved ten cooperatlngt * 
police agencies*^ An eight^test battery was developed for use as tbe predictor. 
Information source: *grant records. |" / 

5» Criterion wasia set of ratings. Information source: recollectiolT^o^^yroject * 

^ director. / , * • \ 

* 1 * 

6. Joint study conducted by civil service commission and fire department. Information 
* Information ^our<re: Coat analysis provided by project director., " 

'7. Research conducted by a consultant. , Study was dpne for and managed by a . t , 

, multi^ jurisdictional consortium* Information source ! records and recollection 
' of project director* * . * 

3. Research conducted by a consultant. Information sourcei recollection of consult* 
ant and fire department* pro ject manager. * ** " 1 . 4 

9* Three part study; (1) concurrent study using white, blac^ and Spanish-speaking male^ 
.transit operators; (2.) predictive study using white, black, and Spanish's peaking male 
applifcajvts whovere subsequently hired; and (3) study using female transit operators. 
Studies were conducted by a hired consultant and managed by one transit company with 
cooperation f^om several other c£ppdnles. jV for the j female transit operator: study \ 
was'not* available. Inf ormrftiorTeburce: grant records; recollection ot grant 
'manager; technical report. v " * - * t** 



v V 1 

10. 1 -Co-operative study conducted by^Pederal government and an outside research , * 

organization, "information source: recollection of principal Investigator and 
technical report. > * 

. * * *■ t * 

11. One 9% a series of four studies done In support -of the construct validity of one 
examination. / Information* source; recollection and records of project members; 



ninatlon. /< 
technical report. 



12. " One of a settles of four studies done in- support, of the construct validity of one 
-examination. Information source: technical report of project costs. 

/ v ■ V . 

10. One of a Aeries of " four studies done In support of the -construct validity of one 

examination. Crlterlon.wss success In training. Information source: recollection 

and records of principal lnvestlgat6rv * 1 f 

* . ft- i • * - * . 
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APPENDIX A (continued) 



V <* 
One of a series of four studies done in support of the construct validity of. one 
examination. Information source: records and recollection' of principal 
investigator; technical report. * 

15. #'s for individual atudies ranged from 75 to 300. . Fftur. studies followed a concurrent 
design; one followed a predictive design. . Three, studies were conducted in the period 
from 1975 to 1976, one was conducted in 1977 , and one * in 1978. Information source: 

. recollection and records of project manager. 

16. Studies were completed in 1973. A consultant was employed to review the six studies. 
Information source: recollection .of project manager. % * 

17. A All data were obtained from records. Therefore, data collection was not included in 

- costs for this study. ^Criterion dsta, performance in advanced training, wevk 
available but needed some modification. Administration of predictor was done by 
teat administration personnel as regular part of their job, and not by researchers. 
* This work was not considered aa part of the study. N *s ranged from 35 to 1,000 for 
any individual occupation. Information source*: principal investigator. 

» * . 
18., Data show^n in Ta'ble 1 «are that of the research report. Forty-three occupational 

clysters were qovered by the study* N 'a ranged from 103 to 5,736 for individual job 
areas studied. All data used were available in recorded form. For the purpoae of 
this cost survey the research organization mad* an estimate of the cost of criterion 
development and data collection and of the cost of predictor data collection. 
Information source: budget records, records and recollection, of principal 
inveatigator, technical report. 



19. 

£0. 



Information source: recorda and recollection of project manager. 



Study produced a two-part content-valid examination including both a written test 
and a physical performance examination. Information source: records and 
recollection of principal inveatigator. ^ 



21. Information source: recollection of project manager. 
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APPENDIX B 
Notes to Entries in Table 2 '* 



Overhead Included only fringe benefit*, estimated at 15% of *the study^e total ^coatji*^' 
Research participant time included time of reaearch aubjettjs a*fld aup&ryiso^^rSter>* ' * 

Re&earcher w and clerical time coata are' co nsul tant's total costs £nd include overhead, 
plus a small amount of travel. Sub ject-nflE&r-e Xpert and research psrtlcipsnt ipos^a 
Include fringe benefits estimated at 30JE'OT^alary ratea. 

Researcher and clerical time included $20,000 in consultant costa plua $10,000 for , 
time of conaortlum personnel. Overhead was -included in estimation coat figures^ but / 
its specific amount ia unknown. 1 • " , • " * ^-~*^L * 



i 



4. Research* participant costs were those of reimbursing police Apartments st ^the rate of 
$75.^0 per officer for research subjects. ; 

5. Only an estimate of total costs was available.' Total does not Include au&ject**ma£te 
expert or reaearch ,participant time. 

" • < . 

6. Beaearcher and clerical time includes 62 benefits and, 1 hX overhead for aalarles of * 
civil service commission personnel and coats' without beneflta or overhead for fire 
department personnel. Subject-mat ter-expfirt and reaearch participant costs do *not 
include overhead or beneflta.. ^ 5 tt \ 

**7. Researcher end clerical time includes $20,000 in total consultant costa .and Tncludes 
overhead. Reaearch participant time included that of both research subjects and ' 
supervisory raters, and does not include overhead, whether overhead ia 'included in 
aubject -matter-expert costa is unknown* 

I > *- ■ 

6. Researcher and clerical time costs are^ total conaultant coats' and include overhead. < 
Research 'participant tine includes that of both research subjects and supervisory * 
ratera, and dfces not include overhead.) The inclusion of overhead in aubject-matter- 
expert coats tjg unknown. 

f ' 
9. Reaearcher an4 clerical time included! clerical time trom the managing tranalt author- 
ity and managerial, technical, and professional time from all participating tranalt 
companies. Overhead Included consultant overhead at 72SE^of conaultant costa -and 
fringe beneflta for transit company employees including reaea^ren participants. 

10. Subject-matter-expert and reaearch participant costs were estimated by the author 
from N *s and time^ spent, using "$24,600 as the .average ctfat of a ataff year including 
overhead, a figure uaed in the Personnel Research and Development Center for budget . 
estimations. Trayel waa estimated by the author from the number of trips,, duration 
of each .trip, and the number of ataff members travelling. Estimated coata of $25 
per diem and $200 per carrier fare were uaed. ^ 

A 

11. Entries 11, 12, 13, .and 14, were part, of a series of -studies done in support of the 
construct validity of one examination, with cost totalling ($66,973. Researcher and 
clerical time, subject-matter-expert, and research participant costa were 1 estimated 
using $24,600 as the average cost of a staff year 47 including overhead. 

" „ ' - T * * * 

12. See note 11. * 
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13. 
14*. 
15. 
16. 
17. 

18. 



19. 



20. 
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APPENDIX' B (cpntlnutfO 



See note 



21, 



See not£ II 

Overhead was .Included in staff-time oxfsts, but Its amount Is unknown, 

Overhead was not included in figiJree.-\ r 

\. . 

Total amount" based on a cost of $60,000 per staff year including overhead. Because 
research participants 'were applicants fpr employment their cost was not/ available 
although potentially large (5,000 applicants at 3 hours each). Computer-time costs 
although* possibly rarge<pere not available. 

1 ' \ ■ C 

SXudy used automated records as its data source* Researchei and clerical time costs 
for, data, analysis include 1002 overhead. Predictor collection costs were estimated 
at^£5 per case and criterion development costs at $10 per case. Specific amounts for 
subject matter 4xi>efta, research partlclp4xit$> data collection^ overhead> travel, and 
and miscellaneous ;vere unavailable. 

Researcher and clerical sta£f time^ costs include fringe benefits. The study involved 
sotre research participant tl-me but this was not available, nor was the cost of print- 
ing.* Miscellaneous costs include only a fee for work done by* an outside consultant. 

Total costs include all operational costs as well as' research and development . 
expenditures. Individual cost element figures were not available. The total 
includes costs foV: .consul tant* 'fees,' salaried and per diem for examination develop- 
ment team, statewide recruitment team, processing 30,500 applications, test printing 
and performance test construction training sessions for the 'administration of the 
two-part examination, developmental work on background investigation and oral 
examination; and test, scoring and production of eligibility test. 

Only an estimate of total costs was available. * I 
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